---
title: "The Effects of Wage Information on Support for Redistributive Spending."
author: 
   - Emily Thorson^['Corresponding author. Assistant Professor, Syracuse University, Syracuse, NY, United States. Email eathorso@syr.edu.']
   - Kris-Stella Trump^['Research Scholar, Johns Hopkins University, Baltimore, Maryland, United States. Email kstrump@jhu.edu.']
thanks: "The authors thank Spencer Piston, Vanessa Williamson, and workshop participants at Syracuse University for useful comments. A version of this project was presented at the 2021 meeting of the American Political Science Association. Replication materials are available on Dataverse ([link https://doi.org/10.7910/DVN/UCUPMF](https://doi.org/10.7910/DVN/UCUPMF)); Study 2 was pre-registered with the Open Science Framework ([link https://osf.io/sqx9f/](https://osf.io/sqx9f/))"
header-includes:
   - \usepackage{setspace}\doublespacing
   - \setlength\parindent{24pt}
   - \usepackage{pdflscape}
   - \usepackage{subfig}
   - \newcommand{\blandscape}{\begin{landscape}}
   - \newcommand{\elandscape}{\end{landscape}}
output: 
  pdf_document:
  number_sections: false
  fig_caption: yes
geometry: margin=1in
fontsize: 12pt
bibliography: wage_info.bib
csl: apa-6th-edition.csl
abstract: "Public support for redistributive policies (e.g. Medicaid) depends in part on the perceived need and deservingness of beneficiaries. However, the average citizen is not well informed about the economic conditions of their fellow citizens. In this article, we explore how information about average earnings of the working poor (a group generally seen as deserving) influences support for redistributive spending. Two survey experiments test whether support for such spending is affected by information about the average incomes of low-wage occupations (e.g. home health aides and retail sales workers). We additionally explore potential mechanisms for this effect, including empathy. An exploratory study finds an effect, but a pre-registered confirmatory study yields substantively small findings with inconsistent significance. Even when participants both receive detailed information about low-wage occupations' salaries and are encouraged to recall people who they know in those jobs, the treatment has no substantial effect. Given the strength of this treatment and the lack of consistent effects, we conclude that interventions providing information about low-income salaries (e.g. in news coverage or interpersonal conversation) are unlikely to have a substantive effect on support for redistribution." 
---

\clearpage

```{r patch, eval=F, include=F}
#as of 12.20.2023 kableExtra and RMarkdown clash - can fix by downloading patched version
# source https://stackoverflow.com/questions/76118194/error-when-loading-kableextra-in-markdown-file-after-updating-to-r-4-3-0
#chunk is set to eval=F because it needs to be run once if issues knitting arise 
library(devtools)
devtools::install_github("kupietz/kableExtra")
```

```{r study 1 setup, include=FALSE}
#Note: this replication file prepared using R version 4.3.3 "Angel Food Cake". All packages updated on 04.30.2024.

#use here package to manage file paths
here::i_am("manuscript.Rmd")

#load some packages, set some options
knitr::opts_chunk$set(echo = TRUE)
library(rmarkdown)
library(haven)
library(tidyverse)
library(stargazer)
library(ggplot2)
library(grid)
library(gridExtra)
library(psych)
library(here)
options(scipen=999, digits=1)

#Read in data
study1 <- read_dta(here("study1_public.dta"))

#Recode variables as needed for analysis
study1 <- study1 %>% 
  mutate(cond = as.numeric(as_factor(cond)),#recoding labelled variables as appropriate
         Q3 = as.numeric(as_factor(Q3)), #income 
         Q30 = as.numeric(as_factor(Q30)),
         Q5 = as.numeric(as_factor(Q5)), #education
         Q1 = as.numeric(as_factor(Q1)), #gender
         Q6 = as.numeric(as_factor(Q6)), #partisanship
         cashier = ifelse(cashier<10000, 10000, cashier), #bottom- and top-code low and high salary estimates
         cashier = ifelse(cashier>80000, 80000, cashier),
         fastfood = ifelse(fastfood<10000, 10000, fastfood),
         fastfood = ifelse(fastfood>80000, 80000, fastfood),
         childcare = ifelse(childcare<10000, 10000, childcare),
         childcare = ifelse(childcare>80000, 80000, childcare),
         aide = ifelse(aide<10000, 10000, aide),
         aide = ifelse(fastfood>80000, 80000, aide),
         retail = ifelse(retail<10000, 10000, retail),
         retail = ifelse(retail>80000, 80000, retail), 
         cashier_bias = cashier - bls_cashier, #create variables for bias in estimates
         fastfood_bias = fastfood - bls_fastfood,
         childcare_bias = childcare - bls_childcare,
         aide_bias = aide - bls_aide,
         retail_bias = retail - bls_retail,
         average_bias = (cashier_bias + fastfood_bias + childcare_bias + aide_bias + retail_bias)/5,
         average_estimate = (cashier + fastfood + childcare + aide + retail)/5,
         hh_income = ifelse(Q3==10, NA, Q3), #income variables for regression, setting "don't know" to NA
         ind_income = ifelse(Q30==10, NA, Q30),
         education = Q5, #just for ease of regression writing
         college = Q5>5,
         female = Q1==2, #dummies for regression
         republican = Q6==2, 
         democrat = Q6==1,
         hc_any = hc_cashier==1 | hc_fastfood==1 | hc_childcare==1 | hc_aide==1 | hc_retail==1, #know someone 
         hc_sum = hc_cashier + hc_fastfood + hc_childcare + hc_aide + hc_retail, #know how many
         cond_info_net = cond==1, #treatment conditions
         cond_net = cond==2,
         cond_info = cond==3,
         cond_ctrl = cond==4,
         info_yes = ifelse(cond_info==1 | cond_info_net==1, 1, 0), #for interaction model
         net_yes = ifelse(cond_net==1 | cond_info_net==1,1,0),
         foodstamps = (as.numeric(Q14_1)-1)/4, #redistribution, rescaled 0-1 and reverse coded where needed
         welfare = (as.numeric(Q14_3)-1)/4,
         childsupp = (as.numeric(Q14_7)-1)/4,
         medicaid = (3-as.numeric(Q16))/2,
         ubi = (Q31_1)/10,
         minwage = (Q31_4)/10,
         tuition = (Q31_5)/10,
         redistr_scale = (foodstamps+welfare+childsupp+medicaid+ubi+minwage+tuition)/7 #using seven variables for scale
         )

#save stats and then implement exclusions
full_n1 <- nrow(study1)

#Exclusions
study1 <- study1 %>%
  filter(surveytime>quantile(surveytime, 0.1)) #Drop the bottom decile of completion times
    
    
#For convenience later, pull out low-income respondents. Go with ~national median split, i.e. hh income <50k
study1_lowinc <- study1 %>% filter(hh_income<6)

#Also split by whether they know any low-income people
study1_nocontacts <- study1 %>% filter(hc_any==0)
study1_contacts <- study1 %>% filter(hc_any>0)
```

```{r study 2 setup, include=FALSE}
#this chunk sets options, imports data, and prepares data for analysis
knitr::opts_chunk$set(echo = F,
                      digits=2)
options(digits=1, scipen=999) #sets digits in printed numbers; scipen reduces scientific notation
#load packages
library(rmarkdown)
library(psych)
library(tidyverse)
library(ggplot2)
library(qualtRics)
library(RColorBrewer)
library(stargazer)
library(kableExtra)
library(dotwhisker)
library(gridExtra)

#load data, prep data for analysis
study2 <- read_csv(here("study2_public.csv")) %>%
  #for network questions, -99 signifies option was displayed but not selected -> set to 0
  mutate(across(starts_with("network_"), ~recode(., '-99' = 0))) %>%
  #set remaining instances (in other survey items) of displayed but unanswered to NA
  mutate(across(where(is.character), ~ na_if(.,"-99")),
         across(where(is.numeric), ~ na_if(.,-99))
         ) %>%
  #merge salary estimates from different conditions
  mutate(
    #attention check pass
    pass_attention = ifelse((attentioncheck_1==1 & attentioncheck_2==1), 1, 0),
    #partisanship
    dem_lean = replace_na(dem_lean, 0),
    rep_lean = replace_na(rep_lean, 0),
    indep_lean = replace_na(indep_lean,0),
    partisan_republican = (1*(dem_lean==1) + 2*(dem_lean==2) + 3*(indep_lean==1) + 4*(indep_lean==3) + 5*(indep_lean==2) + 6*(rep_lean==2) + 7*(rep_lean==1)),
    democrat = partisan_republican <4,
    republican = partisan_republican>4,
    independent = partisan_republican==4,
    #contact dummies
    retail_contact = ifelse(network_1_retail_2==1 | network_1_retail_3==1, 1, 0),
    fastfood_contact = ifelse(network_1_fastfood_2==1 | network_1_fastfood_3==1, 1, 0),
    health_contact = ifelse(network_1_health_2==1 | network_1_health_3==1, 1, 0),
    waiter_contact = ifelse(network_2_waiter_2==1 | network_2_waiter_3==1, 1, 0),
    childcare_contact = ifelse(network_2_childcare_2==1 | network_2_childcare_3==1, 1, 0),
    any_contact = ifelse(retail_contact==1 | fastfood_contact==1 | health_contact==1 | waiter_contact==1 | childcare_contact==1, 1, 0),
    retail_acq = ifelse(network_1_retail_3==1, 1, 0),
    fastfood_acq = ifelse(network_1_fastfood_3==1, 1, 0),
    health_acq = ifelse(network_1_health_3==1, 1, 0),
    waiter_acq = ifelse(network_2_waiter_3==1, 1, 0),
    childcare_acq = ifelse(network_2_childcare_3==1, 1, 0),
    n_acq = retail_acq + fastfood_acq + health_acq + waiter_acq + childcare_acq,
    retail_friend = ifelse(network_1_retail_2==1, 1, 0),
    fastfood_friend = ifelse(network_1_fastfood_2==1, 1, 0),
    health_friend = ifelse(network_1_health_2==1, 1, 0),
    waiter_friend = ifelse(network_2_waiter_2==1, 1, 0),
    childcare_friend = ifelse(network_2_childcare_2==1, 1, 0),
    n_friend = retail_friend + fastfood_friend + health_friend + waiter_friend + childcare_friend,
    n_contact = n_acq + n_friend,
    any_friend = ifelse(n_friend>0, 1,0),
    #recode policy question on min wage so higher numbers indicate the more liberal position
    min_wage_support = (6-support_2)/5,
    #create a 0 to 1 variable that measures overall support for redistribution
    policy_support = (spend_1 + spend_3 + spend_5 + spend_6 + support_1 + min_wage_support + support_3 + support_4)/40,
    #create a 0 to 1 variable that measures support for low-income-worker oriented policies
    lowinc_policy_support = (support_4 + spend_5)/10,
    #create variables for different treatments
    pure_control = ifelse(condition==5 | condition==6, 1, 0),
    info_treatment = ifelse(condition==5 | condition==6, 0, 1),
    contrast_treatment= ifelse(condition==1 | condition==3, 1, 0),
    nocontrast_treatment = ifelse(condition==2 | condition==4, 1, 0),
    estimates_treatment = ifelse(condition==1 | condition==2, 1, 0),
    noestimates_treatment = ifelse(condition==3 | condition==4, 1, 0),
    #in lucid, people who did not want to indicate hhi or education are coded as -3105; set these to NA
    hhi = na_if(hhi, -3105),
    #preserve original hhi
    hhi_categories = hhi,
    #recode income to thousands USD, taking midpoint of each category
    hhi = 7.5*(hhi==1) + 17.5*(hhi==2) + 22.5*(hhi==3) + 27.5*(hhi==4) + 32.5*(hhi==5) + 37.5*(hhi==6) + 42.5*(hhi==7) + 47.5*(hhi==8) + 52.5*(hhi==9) + 57.5*(hhi==10) + 62.5*(hhi==11) + 67.5*(hhi==12) + 72.5*(hhi==13) + 77.5*(hhi==14) + 82.5*(hhi==15) + 87.5*(hhi==16) + 92.5*(hhi==17) + 97.5*(hhi==18) + 112.5*(hhi==19) + 137.5*(hhi==20) + 162.5*(hhi==21) + 187.5*(hhi==22) + 225*(hhi==23) + 275*(hhi==24),
    education = na_if(education, -3105),
    #recode lucid's partisanship indicator to democrat, republican
    lucid_dem = political_party<4 |  political_party==6,
    lucid_indep = political_party == 4 | political_party==7,
    lucid_rep = political_party>7 | political_party==5,
    #dummies for demographics
    white = ethnicity==1,
    black = ethnicity==2,
    hispanic_dummy = hispanic != 1,
    college = education > 4,
    male = gender==1,
    #the next two for easier plotting
    contrast_plot = ifelse(contrast_treatment==1, 1, ifelse(nocontrast_treatment==1, 2,3)), #1 = hi contr, 2 = lo contr, 3 = ctrl
    est_plot = ifelse(estimates_treatment==1, 1, ifelse(noestimates_treatment==1, 2,3)), #1 = est, 2 = no est, 3 = ctrl
    #salary estimates
    #for occupation-specific analysis, combine the hi and low condition salary estimates for each occupation into one variable each
    #this step not needed for high income estimates as these are only asked in condition 1
    retail_est = ifelse(condition==1, retail_est_hi, retail_est_lo),
    fastfood_est = ifelse(condition==1, fastfood_est_hi, fastfood_est_lo),
    health_est = ifelse(condition==1, health_est_hi, health_est_lo),
    waiter_est = ifelse(condition==1, waiter_est_hi, waiter_est_lo),
    childcare_est = ifelse(condition==1, childcare_est_hi, childcare_est_lo),
    #count how many occupation guesses each respondent made (depends on trtmnt condition and whether they skipped any responses)
    n_lowinc_estimates = !is.na(retail_est) + !is.na(fastfood_est) + !is.na(health_est) + !is.na(waiter_est) + !is.na(childcare_est),
    n_highinc_estimates = !is.na(dentist_est) + !is.na(software_est) + !is.na(lawyer_est),    
    #winsorize low-income estimates at 100,000
    retail_est = ifelse(retail_est>=100000, 100000, retail_est),
    fastfood_est = ifelse(fastfood_est>=100000, 100000, fastfood_est),
    health_est = ifelse(health_est>=100000, 100000, health_est),
    childcare_est = ifelse(childcare_est>=100000, 100000, childcare_est),
    waiter_est = ifelse(waiter_est>=100000, 100000, waiter_est),
    #did people over/underestimate low wage occupations' incomes
    retail_overest_dummy = as.numeric(retail_est > 29360),
    fastfood_overest_dummy = as.numeric(fastfood_est > 23250),
    health_overest_dummy = as.numeric(health_est > 24200),
    waiter_overest_dummy = as.numeric(waiter_est > 26800),
    childcare_overest_dummy = as.numeric(childcare_est > 25460),
    #was there any over-estimate? note that number guessed varied by condition so different # chances to get it wrong
    lowinc_overest_dummy = retail_overest_dummy==1 | fastfood_overest_dummy==1 | health_overest_dummy==1 | waiter_overest_dummy==1 | childcare_overest_dummy==1,
    #what was the amount of over/underestimation
    retail_overest_amt = retail_est - 29360,
    fastfood_overest_amt = fastfood_est - 23250,
    health_overest_amt = health_est - 24200,
    waiter_overest_amt = waiter_est - 26800,
    childcare_overest_amt = childcare_est - 25460
    ) %>%
  rename(duration_sec = "Duration in seconds")


#save some stats and then implement exclusions
full_n <- nrow(study2)
no_consent <- sum(study2$consent!=4)

#first exclusion: drop non-consenters
study2 <- study2 %>% 
  filter(consent==4)

#how many failed attention check
fail_attention_check <- nrow(study2) - sum(study2$pass_attention==1, na.rm = T)

#pre-registered exclusion 1: attention check
study2 <- study2 %>% 
  #attention check 
  filter(pass_attention==1)

#pre-registered exclusion 2: fast completions 
study2 <- study2 %>% 
  group_by(condition) %>%
  #2% fastest in each condition get dropped
    filter(duration_sec >= quantile(duration_sec, probs=0.02)) %>%
  ungroup()

#break out conditions 1 and 2 for easier reporting 
study2_cond1_hi <- study2 %>%
  filter(condition==1)
study2_cond2_lo <- study2 %>%
  filter(condition==2)

#break out people who have at least one contact
study2_contacts <- study2 %>% 
  filter(any_contact==1)

#save a version as data frame for easy in-text reporting of means etc
study2_df <- as.data.frame(study2)
study2_contacts_df <- as.data.frame(study2_contacts)
```

# The Effects of Wage Information on Support for Redistributive Spending

## Introduction

Many Americans may be unaware of how many of their fellow citizens, including people they know or interact with regularly, are members of the working poor. The number of people working full-time but barely getting by is at record levels: seventy percent of enrollments in major public benefits programs are from families with at least one full-time worker [@governmentaccountabilityoffice2020], and four of the five largest occupations in the U.S. pay under \$30,000 a year [@bureauoflaborstatistics2021].

However, the public is generally not well informed when it comes to economic statistics about incomes and inequality [@gimpelson2018; @pontusson2020], and it is plausible that many Americans underestimate the prevalence of the working poor. In 2022, for example, the median respondent to a YouGov poll guessed that the federal minimum wage was \$9.88 per hour (it was \$7.25 at the time), and thought that it should be \$14.88 per hour [@orth2022]. A similar pattern of over-estimation occurred for state minimum wages, suggesting that over-estimation is not driven by local cost of living differences.

In this paper, we evaluate whether information about low-wage incomes can affect support for redistributive spending, including spending specifically targeted at the working poor. We hypothesize that this type of information may be particularly likely to influence attitudes that are otherwise hard to move with information experiments [@ciani2021], because low-wage salaries in particular provide a relatable and easily understood benchmark [@moniz2022] and because the information pertains to a group who tend to be seen as the "deserving poor" [@katz1989]. To the best of our knowledge, the effect of this information on support for redistributive spending has not previously been studied.

We present the results of two experiments in which we test this prediction. We find that, despite promising results in a pre-test, a pre-registered and well-powered replication does not yield substantively significant effects. We additionally test three potential mechanisms for any potential effect, asking whether effects may be enhanced by contrasting the working poor with the rich, being made aware of one's misperceptions regarding the true incomes of the working poor, or activating empathy for the working poor. None of these factors significantly affect the results. We ultimately conclude that although many people do hold misperceptions about low-wage salaries, correcting these misperceptions likely does not have a substantive or consistent effect on redistributive attitudes.

## How information shapes support for redistributive spending

Experimental studies exploring how information about the income distribution affects support for redistribution have become common enough to be the subject of a recent meta-analysis [@ciani2021]. These studies have tested the effect of accurate information about the income distribution [@zilinsky2014; @kuziemko2015; @trump2018a] as well as one's own position within it [@karadja2017; @hoy2019a]. Results have been mixed; the meta-analysis by @ciani2021 concludes that while this type of information can increase the public's concern about economic outcomes, it has at best small effects on support for redistributive spending.

In most of these experimental studies, the information given to participants consists of aggregate figures (e.g. overall levels of inequality). However, many people have difficulty processing complex numerical quantities [@ansolabehere2012; @eriksson2012], a necessary first step in using the information to inform their attitudes. Even if it is understood and processed, numeric information may not change opinions, because the numbers may not be connected to relevant normative concerns [@lawrence2014; @moniz2022]. In this paper, we test the impact of a specific type of economic information: the wages of low-income occupations. Specifically, we test the hypothesis that providing people with information about incomes at the low end of the income distribution can change policy attitudes.

Information about the actual salaries of workers in low-income occupations has several desirable qualities in light of past research. First, occupational wages have an simple basis of comparison: one's own earnings, which should facilitate interpreting the information and connecting it to normative concerns [@moniz2022]. Second, this information explicitly pertains to the *working* poor, thus increasing the probability that the targets are seen as deserving [@petersen2011]. Consistent with these expectations, @newman2022 finds that information about the prevalence of the working poor reduces belief in the American dream, though his experiment differs from ours in that information about the working poor is presented by describing the existence of the working poor in a context of inequality (rather than mentioning individual jobs and salaries), and the outcome variable is belief in the American dream (rather than support for specific policies). Building on this prior literature, we therefore expect that information about low-wage salaries can affect support for redistributive spending, even as more aggregate inequality statistics do not have such an effect.

**H1:** Information about the salaries of low-income workers increases support for redistributive policy.

In addition to our main hypothesis, we also explore whether any information effects are augmented by one of three mechanisms. First, in keeping with the observation that easily accessible benchmarks facilitate interpretation and connection to normative concerns, we ask whether contrasting high and low incomes increases the effect of the treatment. Therefore, one of our treatment conditions provides an explicit point of contrast by specifying the incomes of a few high-earning occupations, in addition to the information about the incomes of low-wage occupations.

**RQ1:** Does the additional provision of information about the salaries of high-income workers increase the impact of the information treatment?

Second, the effect of learning correct information about salaries may be more salient if the respondent is made aware of their (otherwise implicit) over-estimation. One of our treatment conditions asks respondents to estimate the incomes of low-wage earners before they see the information treatment. Receiving correct information immediately after submitting their own guesses serves to highlight any underestimation.

**RQ2:** Does being asked to provide estimates of low-income workers' salaries before receiving the information treatment increase the impact of the information treatment?

Third, bringing to mind someone who works but nonetheless struggles to make ends meet may change policy attitudes by activating empathetic concern [@piston2018; @batson2009]. Empathetic concern is elicited more easily for some groups than for others. People tend to believe that people like them are especially deserving [@harth2008], and find it easier to feel empathy towards people who are similar to them [@lane2001]. This tendency translates to attitudes towards redistributive spending, as people tend to be more supportive of such spending when they believe that the recipients are deserving [@goren2009; @cook1992], engender feelings of compassion [@petersen2012a], and are similar to themselves [@luttmer2001]. Given these patterns, we might expect support for redistribution to increase when people believe it will benefit people whom they know personally. We probe this possibility by separately analyzing respondents who report personally knowing someone in a low-wage occupation.

**RQ3:** Is the treatment effect larger among respondents who know someone in one of the low-wage occupations in the experiment?

# Exploratory study

The exploratory study, conducted in 2014, explored how (1) receiving information about low-wage incomes and (2) knowing people in low-wage jobs affects support for redistributive spending. The experiment included an information treatment, in which participants were asked to estimate the average yearly salary for a full-time worker in a selection of high- and low-wage occupations, and then received information about the real salaries in those occupations. The experiment also included a network treatment, in which participants were asked whether and how closely they knew anyone in the same selection of occupations.

1006 respondents recruited via Survey Sampling International were randomly assigned to complete the information treatment, the network treatment, neither, or both. Respondents were then asked about their support for several policies, including five aimed at helping low-income people. These ranged from moderate policies (child care for low income families) to more extreme (a guaranteed income from the U.S. government). A more detailed description of this experiment is available in the Online Appendix.

The results (also available in the Online Appendix) show that information affected attitudes: people exposed to information became more supportive of redistributive spending While the effect occurred among all respondents, it was largest among people who knew someone in a low-income occupation (effect size 0.08 on a 0-1 scale, equivalent to 0.4 standard deviations).

The follow-up study described below was designed to to replicate these findings, address some weaknesses in the exploratory study, and investigate plausible mechanisms based on the initial results. We pre-registered the replication to guard against accepting a chance finding as real.

# Main study

The main experiment evaluated the hypothesis that information about the average yearly salary for low-wage occupations increases support for redistribution. This study was pre-registered with the Open Science Framework prior to any data collection ([link https://osf.io/sqx9f/](https://osf.io/sqx9f/)). The experiment was a 2 (contrast with high-wage occupations/no contrast with high-wage occupations) x 2 (participants provided estimates prior to treatment/participants did not provide estimates prior to treatment) plus pure control design.

The full list of conditions is shown below:

1.  High contrast/estimates: Asked to estimate incomes of high- and low-wage occupations, then were shown correct incomes for high- and low-wage occupations
2.  No contrast/estimates: Asked to estimate incomes of low-wage occupations, then were shown correct incomes for low-wage occupations
3.  High contrast/no estimates: Shown correct incomes for high- and low-wage occupations
4.  No contrast/no estimates: Shown correct incomes for low-wage occupations
5.  Pure control

Participants in all four treatment conditions saw accurate information about the salaries of low-wage occupations, but the presentation varied according to treatment group. The particpants in the "estimates" conditions (1 and 4) were asked to offer their own estimates (in an open-ended format) prior to being shown the correct numbers. Specifically, they were told \`\`Next you will see five common jobs in the U.S., along with a brief description of the job. For each of the five jobs, we'd like you to give your best guess: what is the average yearly income for a full-time worker in this job? By full-time worker, we mean a person who works at least 8 hours a day, five days a week.''

The number of occupations a respondent was asked about was kept constant at five. The occupations in the "no contrast" conditions were child care workers, fast food workers, home health aides, retail salespeople, and waiters/waitresses. The occupations in the "high contrast" condition included three randomly selected from that list, plus two high-income occupations (randomly selected from dentists, lawyers, and software developers). Participants were shown the correct incomes in chart format. Figure 1 shows the information treatments for the "high-contrast" conditions (1 and 3) and "no-contrast" conditions (2 and 4).

Finally, all participants answered a battery of questions assessing their preferences regarding government spending on four different existing programs (food stamps, welfare programs, child care for low-income families, Medicaid) as well as their support for four novel policies: lowering the minimum wage (reverse-coded), a guaranteed income from the U.S. government, free college education, and expanding the Earned Income Tax Credit (described as a tax credit for low-income workers).

## Results

```{r cronbachs alpha for policy responses, include=F}
#did the policy items hang together
policy_items_spend <- study2 %>% select("spend_1", "spend_3", "spend_5", "spend_6")
policy_items_support <- study2 %>% select("support_1", "min_wage_support", "support_3", "support_4")
policy_items <- study2 %>% select("spend_1", "spend_3", "spend_5", "spend_6", "support_1", "min_wage_support", "support_3", "support_4")
policy_alpha <- psych::alpha(policy_items)
#full results
policy_alpha
#to only call the cronbach's alpha estimate
policy_alpha$total$raw_alpha
```

```{r set digits}
options(digits = 1)
```

We fielded the survey in August 2021 through the survey company Lucid, recruiting a convenience sample of `r full_n` respondents who were residents of the United States. As pre-registered, respondents who failed an early attention check were not allowed to complete the survey, and we dropped the 2% fastest responders in each condition; after these steps the final sample consisted of `r nrow(study2)` respondents. The median age of respondents in the final sample was `r median(study2$age_1, na.rm=T)`. `r round(sum(study2$gender==1)/nrow(study2)*100, 0)`% were male, `r round(sum(study2$ethnicity==1)/nrow(study2)*100, 0)`% were white, `r round(sum(study2$ethnicity==2)/nrow(study2)*100, 0)`% were Black, and `r round(sum(study2$hispanic!=1)/nrow(study2)*100, 0)`% were Hispanic of any race. `r round(sum(study2$education>4, na.rm=T)/nrow(study2)*100, 0)`% had a college degree. The median household income was in the \$40,000 - \$45,000 range. `r round(sum(study2$democrat==1)/nrow(study2)*100, 0)`% identified as Democrats (including leaners) and `r round(sum(study2$republican==1)/nrow(study2)*100, 0)`% identified as Republicans (including leaners).

Over-estimation of the salaries of low-income workers was common. In the low-contrast condition (where questions about high salaries could not create a mental anchor), `r round(sum(study2_cond2_lo$lowinc_overest_dummy, na.rm=T)/nrow(study2_cond2_lo)*100, 0)`% of respondents over-estimated the pay of at least one of the five low-income occupations (in the high contrast condition, all respondents over-estimated at least one low-income salary). On average, respondents in the low-contrast condition over-estimated annual salaries by \$`r round((mean(study2_cond2_lo$health_overest_amt, na.rm=T) + mean(study2_cond2_lo$retail_overest_amt, na.rm=T) + mean(study2_cond2_lo$fastfood_overest_amt, na.rm=T) + mean(study2_cond2_lo$childcare_overest_amt, na.rm=T) + mean(study2_cond2_lo$waiter_overest_amt, na.rm=T))/5, 0)`.

```{r set digits 2}
options(digits = 2)
```

The dependent variable was a linear 0-1 scale based on support for spending on redistributive policies (Cronbach's Alpha `r policy_alpha$total$raw_alpha`). A planned secondary analysis using only those policies with the most direct relevance to low-income workers (spending more on childcare for low-income families and increasing the Earned Income Tax Credit for low-income workers) yielded similar results as those reported below (results available in the Online Appendix).

## Main hypothesis: effect of information

```{r hypothesis test primary outcome, include=F, eval=T}
#hypothesis: Information about the salaries of low-income workers increases support for redistributive policy.
#primary DV: average of eight policy items
h1_reg_prereg <- lm(policy_support ~ info_treatment + democrat + independent + age_1 + white + hispanic_dummy + hhi + college + male, data=study2)
```

```{r hypothesis test primary outcome with interaction, include=F, eval=T}
#hypothesis: Information about the salaries of low-income workers increases support for redistributive policy in differential ways depending on respondent partisanship.
#primary DV: average of eight policy items
h1_reg_partisanship <- lm(policy_support ~ info_treatment*democrat + independent + age_1 + white + hispanic_dummy + hhi + college + male, data=study2)
```

Figure \ref{fig:H1} shows the effect of receiving information about low-income workers' incomes on redistributive attitudes, using our pre-registered specification with covariates. The information treatment consists of groups 1-4, with the control group serving as the reference category. This overall treatment effect is statistically significant (p = `r summary(h1_reg_prereg)$coefficients["info_treatment",4]`) but substantially modest with a point estimate of `r summary(h1_reg_prereg)$coefficients["info_treatment",1]` on a 0 - 1 scale (mean across all conditions `r mean(study2$policy_support, na.rm=T)`, standard deviation `r sd(study2$policy_support, na.rm=T)`). Tables with regression results underlying all figures are available in the Online Appendix. While we pre-registered the specification with covariate controls, it is additionally illustrative that raw differences between experimental conditions are even smaller and statistically insignificant, as shown in Figure \ref{fig:raw_means}. The effects do not vary by respondent partisanship (the details of this exploratory analysis are reported in the Online Appendix).

## Effect of variation in information presentation

```{r rq 1 effect of contrast, include=F, eval=T}
rq1_reg_ctrls <- lm(policy_support ~ contrast_treatment + nocontrast_treatment  + democrat + independent + age_1 + white + hispanic_dummy + hhi + college + male, data=study2)
```

Figures \ref{fig:RQ1} and \ref{fig:RQ2} address the effects of contrasts with high income salaries, and of being asked to provide salary estimates. First, RQ1 asks whether the treatment is more effective when a contrast with high income workers is provided. Figure \ref{fig:RQ1} shows coefficients from a regression with the same covariates as in the pre-registered analysis for H1. While the small effect of information persists, there is no difference between the "contrast" and "no contrast" conditions, which is what this pre-registered test was intended to explore. In other words, information about low-income salaries increased support for redistribution, but the effect did not vary with the presence of a contrast with high-income occupations.

```{r rq2 effect of estimates, include=F, eval=T}
rq2_reg_ctrls <- lm(policy_support ~ estimates_treatment + noestimates_treatment  + democrat + independent + age_1 + white + hispanic_dummy + hhi + college + male, data=study2)
```

RQ2 explores the effect of being asked to provide estimates prior to seeing the actual salaries. Figure \ref{fig:RQ2} reports the results from a regression with covariates, following the pre-registered specification. The results are similar to RQ1: while the small information effect persists, the difference between the conditions with / without estimate prompts is not significant. As before, a comparison of treatment group means also yields insignificant results.

## Moderation by low-income contacts

RQ3 asked whether the effect of information was stronger for people who knew at least one person in low-income occupations. `r round(nrow(study2_contacts)/nrow(study2)*100, 0)`% of the sample (N=`r nrow(study2_contacts)`) reported knowing at least one person in a low-wage occupation. Figures \ref{fig:H1}-\ref{fig:RQ2} all show coefficient estimates for this subset of the sample, in addition to estimates for the full sample. None of the treatment effects are statistically significant among this subset of respondents. Additionally, no effects are significantly different from the effects observed in the full sample. The pattern of null results also persists when comparing raw means by treatment condition among this subset of respondents. Finally, a planned exploratory analysis using a one-item trait empathy scale [@konrath2018a] reveals neither a main effect of empathy on support for redistribution, nor an interaction effect between trait empathy and information about low-income wages.

# Discussion

For the most part, our experiment did not replicate the results found in the exploratory study. On the one hand, the main effect of information is statistically significant, suggesting that information may indeed have a minor effect on attitudes. On the other hand, the point estimates are in the 0.01 - 0.02 range for a 0 - 1 variable with standard deviation 0.16. While our pre-registered specification with covariates yields significant effects, we are also concerned about the lack of any statistically or substantively significant effects in comparisons of raw means between experimental conditions. Additionally, none of our pre-registered research questions about contrast effects, providing estimates, or knowing someone in a low-income occupation affected the outcome.

This effect is substantively very small, despite a well-powered experiment whose design includes several features aimed at strengthening the treatment and/or identifying effects among susceptible subgroups. First, the treatment was accessible: salary information (in contrast to aggregate information about the income distribution) is a familiar quantity with a clear benchmark for most Americans [@ansolabehere2012]. Second, the treatment was strong: respondents were provided information about the salaries of multiple low-income jobs in an easy-to-process infographic format. In addition, we made several deliberate design choices to explore potential mechanisms for our hypothesized information effect. First, some respondents were asked to estimate salaries prior to being given the correct information, thereby making any differences between the respondents' expectations and the actual salaries even more salient. Second, respondents were asked whether they personally knew anyone in a low-wage job, with the goal of investigating whether a personal connection to a low-wage worker would heighten the impact of salary information. Finally, we included a measure of empathy, again expecting that respondents high in empathy may have stronger reactions to the information. We find no differences in treatment by any of these conditions or characteristics, which we interpret as additional evidence that the information treatment is ineffective even among people whom we might expect to be particularly susceptible to it.

There are several potential reasons why the exploratory findings may have failed to replicate. The first is that the initial findings may have occurred by chance. This risk is precisely what pre-registered replications are intended to mitigate. The fact that the null occurred in the pre-registered, better powered study with an improved design makes us inclined to accept the null result over the exploratory positive result. However, we would be remiss to not discuss other potential reasons for the conflicting findings.

Another explanation for the different results is time: the exploratory study was conducted in 2014, during a strong economy, while this study took place seven years later during a time of economic upheaval. The difficulty of making ends meet was a major news topic throughout the COVID pandemic, featuring stories about people (especially low-income essential workers) facing difficulties affording daycare, food, and even medical treatment. In other words, respondents may have been "pre-treated" with strong doses of information about the day-to-day challenges of being a low-income worker in America [@druckman2012]. In support of this interpretation, we find that over-estimation of low-wage salaries was less prevalent in the second study compared to the first. In addition, when so many people are themselves struggling it may be more difficult to muster additional sympathy for others.

Finally, the samples also differed between the two experiments. The sample for the second study was more Democratic, less wealthy, and less educated than the exploratory sample. These differences may contribute to a "ceiling effect" in which there was simply less room to move attitudes on redistributive spending Alternatively, the main sample may have been better at guessing low-income salaries because of these demographic differences, in addition to the different political context discussed above.

These potential explanations for our diverging findings highlight the importance of replicating experimental findings across time and context. To consider an experimental effect an established finding, we should expect it to be robust to at least a certain amount of variation in the political environment or the demographic composition of the sample. The fact that this effect did not survive such variation suggests to us that it is at least not a consistent effect, even if it remains possible that it arises under specific circumstances.

We therefore conclude that information about how little low-wage workers earn does not have a consistent effect on support for redistributive spending. Even if small effects exist, these are not substantial, nor are they affected by contrasts with high salaries, highlighting misperceptions, or knowing someone in a low-wage occupation. This finding is consistent with a growing body of literature showing the limited effects of factual information on attitudes. For example, @coppock2023conceptual find that while fact-checks are consistently effective in altering factual beliefs on a wide range of issues, they fail to affect downstream attitudes, and @bisgaard2019getting shows how people engage in "partisan interpretation" of factual information about the economy. Although people systematically over-estimate how much low-wage jobs pay, correcting those misperceptions has at best minimal effects on support for redistributive spending, even among respondents who personally know people in those jobs.

\newpage

```{r information treatment figure, echo=FALSE, out.width="70%", fig.align = "center", fig.cap=c("Information treatments"), fig.subcap =c("High contrast condition", "Low contrast condition"), fig.ncol=1}

knitr::include_graphics(c("hicontrast.png"))

knitr::include_graphics(c("lowcontrast.png"))
```

\newpage

```{r coefficient plot H1 all respondents vs at least one contact, fig.cap="\\label{fig:H1}Main results: impact of information on support for redistribution"}
options(digits=3)
#table includes pre-registered regression test of H1 for primary and secondary outcome
my_colors <- c("grey", "black") 
h1_reg_prereg <- lm(policy_support ~ info_treatment + democrat + independent + age_1 + white + hispanic_dummy + hhi + college + male, data=study2)
h1_reg_prereg_contacts <- lm(policy_support ~ info_treatment + democrat + independent + age_1 + white + hispanic_dummy + hhi + college + male, data=study2_contacts)
dwplot(list(h1_reg_prereg_contacts, h1_reg_prereg)) %>%
  relabel_predictors(
        c(
            info_treatment = "Information treatment",
            democratTRUE = "Democrat",
            independentTRUE = "Independent",
            age_1 = "Age (years)",
            whiteTRUE = "White",
            hispanic_dummyTRUE = "Hispanic",
            hhi  = "Household income \n(thousands USD)",
            collegeTRUE = "College educated",
            maleTRUE = "Male"
        )
    ) +
  theme_bw() +
  ggtitle("Effect of information on support for redistribution (0-1)", subtitle = "Main effect: pre-registered specification") +
  geom_vline(xintercept = 0,
            colour = "grey60",
            linetype = 2) +
 theme(legend.title = element_blank(),
       legend.position = "bottom"
       ) +
  scale_color_manual(values = my_colors,
                  labels = c("At least one low-income contact", "All respondents"),
                   guide = guide_legend(reverse = TRUE)
                   )

```

\newpage

```{r figure primary outcome all conds grayscale, echo=F, include=T, warning=F, fig.cap="\\label{fig:raw_means}Results by information treatment types"}
my_colors <- c("#636363", "#bdbdbd", "#f7f7f7")
all_conds_plot <- ggplot(study2, aes(x=factor(contrast_plot), y=policy_support, fill=factor(est_plot))) +
  geom_bar(stat="summary", fun.data="mean_se",position='dodge')+
  geom_errorbar(stat="summary", fun.data="mean_se", fun.args=list(mult=1.96), position=position_dodge(0.9), width=0.2) +
  coord_cartesian(ylim = c(0,1),
                  xlim = c(1,3)) +
  ylab("Redistributive policy\nsupport") +
  xlab(NULL) + 
  ggtitle("Effect of information types \n on policy support") +
  theme(axis.text = element_text(size = 12),
        plot.title = element_text(hjust = 0.5)#,
        #axis.title.y = element_text(angle=0, vjust=0.5)
        ) +
  scale_x_discrete(breaks=c("1","2","3"),
        labels=c("High contrast", "Low contrast", "Control")) +
  scale_fill_manual(values = my_colors,
                         breaks=c("1","2", "3"),
                         name=NULL,
                         labels=c("Asked estimates", "Not asked estimates", "Control")
                    ) +
  theme(text = element_text(size = 12)) 
all_conds_plot
```

\newpage

```{r coefficient plot RQ1 all respondents vs at least one contact, fig.cap="\\label{fig:RQ1} Impact of high-salary contrast on support for redistribution"}
options(digits=3)
#controls the same as in pre-registered H1 specification
my_colors <- c("grey", "black") 
rq1_reg_ctrls <- lm(policy_support ~ contrast_treatment + nocontrast_treatment  + democrat + independent + age_1 + white + hispanic_dummy + hhi + college + male, data=study2)
rq1_reg_ctrls_contacts <- lm(policy_support ~ contrast_treatment + nocontrast_treatment  + democrat + independent + age_1 + white + hispanic_dummy + hhi + college + male, data=study2_contacts)
dwplot(list(rq1_reg_ctrls_contacts, rq1_reg_ctrls)) %>%
  relabel_predictors(
        c(contrast_treatment = "Contrast",
            nocontrast_treatment = "No contrast",
            democratTRUE = "Democrat",
            independentTRUE = "Independent",
            age_1 = "Age (years)",
            whiteTRUE = "White",
            hispanic_dummyTRUE = "Hispanic",
            hhi  = "Household income \n(thousands USD)",
            collegeTRUE = "College educated",
            maleTRUE = "Male"
        )
    ) +
  theme_bw() +
  ggtitle("Effect of contrast treatment on support for redistribution (0-1)", subtitle = "Comparison group: pure control") +
  geom_vline(xintercept = 0,
            colour = "grey60",
            linetype = 2) +
 theme(legend.title = element_blank(),
       legend.position = "bottom"
       ) +
  scale_color_manual(values = my_colors,
                  labels = c("At least one low-income contact", "All respondents"),
                   guide = guide_legend(reverse = TRUE)
                   )

```

\newpage

```{r coefficient plot RQ2 all respondents vs at least one contact, fig.cap="\\label{fig:RQ2}Impact of asking salary estimates on support for redistribution"}
options(digits=3)
#controls the same as in pre-registered H1 specification
my_colors <- c("grey", "black") 
rq2_reg_ctrls <- lm(policy_support ~ estimates_treatment + noestimates_treatment  + democrat + independent + age_1 + white + hispanic_dummy + hhi + college + male, data=study2)
rq2_reg_ctrls_contacts <- lm(policy_support ~ estimates_treatment + noestimates_treatment  + democrat + independent + age_1 + white + hispanic_dummy + hhi + college + male, data=study2_contacts)
dwplot(list(rq2_reg_ctrls_contacts, rq2_reg_ctrls)) %>%
  relabel_predictors(
        c(estimates_treatment = "Estimates",
            noestimates_treatment = "No estimates",
            democratTRUE = "Democrat",
            independentTRUE = "Independent",
            age_1 = "Age (years)",
            whiteTRUE = "White",
            hispanic_dummyTRUE = "Hispanic",
            hhi  = "Household income \n(thousands USD)",
            collegeTRUE = "College educated",
            maleTRUE = "Male"
        )
    ) +
  theme_bw() +
  ggtitle("Effect of estimates treatment on support for redistribution (0-1)", subtitle = "Comparison group: pure control") +
  geom_vline(xintercept = 0,
            colour = "grey60",
            linetype = 2) +
 theme(legend.title = element_blank(),
       legend.position = "bottom"
       ) +
  scale_color_manual(values = my_colors,
                  labels = c("At least one low-income contact", "All respondents"),
                   guide = guide_legend(reverse = TRUE)
                   )

```

\newpage

# References {.unnumbered}

::: {#refs}
:::
